Repository navigation
validate_and_wait: poll the validation job, fail fast on failed runs - #8
Merged
Merged
Conversation
…ions The backend's current line (MasterOneVersion) answers addValidationJob with the new job's id (the bare `44570` seen live). start_validation now returns it (None for an empty body) and takes optional dataflow_id/provider_id, which enable the route's ApiKey check there. validate_and_wait polls the job via pollForJobStatus, so a FAILED/REFUSED/CANCELED run raises JobError with the job's status and info at once instead of timing out; a FINISHED job with an info text warns. Without a job id, or when polling is refused (ApiKey-only on MasterDataLakes), it falls back to the notification wait, which now also ends on VALIDATION_REFUSED/CANCELED/FAILED_* notifications. Dataset.validate passes the handle's provider_id. Read from the backend source; not yet run live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fall back to notifications also when the job poll answers 401 or a rejected-credential 500; a poll timeout names the job, dataset and the caller's timeout; a FINISHED-with-note result carries the note in attrs['validation_note']; the job id parser accepts only ASCII digits (also JSON-quoted). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Until now,
validate_and_waitwaited for aVALIDATION_FINISHED_EVENTuser notification. A failed run creates no such notification, so the helper always ran into its timeout (600 s by default).The backend's current line (branch
MasterOneVersion,JobControllerImpl#addValidationJob) returns the new job's id. That is the bare number (44570) we saw live and didn't understand.Changes
validation.start_validation()(sync and async):Nonewhen the body is empty (developandMasterDataLakesanswervoid);dataflow_idandprovider_id, which enable the route's ApiKey branch there.easy.validate_and_wait()polls the job throughpollForJobStatus, the same route our import and export helpers use:JobErrorat once, withstatusand the job'sinfo(for example "Validation failed due to system error…");infotext (for example "There are canceled validation tasks…"): aRuntimeWarning.MasterDataLakes), it falls back to waiting for the notification. That wait now also stops onVALIDATION_REFUSED_EVENT,VALIDATE_AS_PROVIDER_REFUSED_EVENT,VALIDATION_CANCELED_EVENTandVALIDATION_FAILED_*_EVENT.provider_id(a reporter ApiKey needs it on both the start and the poll).Dataset.validate()passes the handle's own.Known gap (backend): one internal-error path in
validateDataSetDataonly marks the dataset as having a validation error. It changes neither the job nor the notifications, so that case still ends in the timeout.Testing: mocked tests only, covering a job that finishes, fails, or finishes with a note, a refused poll falling back to notifications, a failure notification, and the new resource parameters and return value. This is not yet run live; one sandbox run, including a run meant to fail, would confirm it.
Independent of #7. Whichever merges second needs a small CHANGELOG merge (both add to "Unreleased").
🤖 Generated with Claude Code